Discontinuity induced bifurcations of non-hyperbolic cycles in nonsmooth systems

We analyse three codimension-two bifurcations occurring in nonsmooth systems, when a non-hyperbolic cycle (fold, flip, and Neimark-Sacker cases, both in continuous- and discrete-time) interacts with one of the discontinuity boundaries characterising the system's dynamics. Rather than aiming at a complete unfolding of the three cases, which would require specific assumptions on both the class of nonsmooth system and the geometry of the involved boundary, we concentrate on the geometric features that are common to all scenarios. We show that, at a generic intersection between the smooth and discontinuity induced bifurcation curves, a third curve generically emanates tangentially to the former. This is the discontinuity induced bifurcation curve of the secondary invariant set (the other cycle, the double-period cycle, or the torus, respectively) involved in the smooth bifurcation. The result can be explained intuitively, but its validity is proven here rigorously under very general conditions. Three examples from different fields of science and engineering are also reported

Task-Oriented Active Sensing via Action Entropy Minimization

This work is licensed under a Creative Commons Attribution 4.0 International License.In active sensing, sensing actions are typically chosen to minimize the uncertainty of the state according to some information-theoretic measure such as entropy, conditional entropy, mutual information, etc. This is reasonable for applications where the goal is to obtain information. However, when the information about the state is used to perform a task, minimizing state uncertainty may not lead to sensing actions that provide the information that is most useful to the task. This is because the uncertainty in some subspace of the state space could have more impact on the performance of the task than others, and this dependence can vary at different stages of the task. One way to combine task, uncertainty, and sensing, is to model the problem as a sequential decision making problem under uncertainty. Unfortunately, the solutions to these problems are computationally expensive. This paper presents a new task-oriented active sensing scheme, where the task is taken into account in sensing action selection by choosing sensing actions that minimize the uncertainty in future task-related actions instead of state uncertainty. The proposed method is validated via simulations

Human-like Rewards To Train A Reinforcement Learning Controller For Planar Arm Movement

High-level spinal cord injury (SCI) in humans causes paralysis below the neck. Functional electrical stimulation (FES) technology applies electrical current to nerves and muscles to restore movement, and controllers for upper extremity FES neuroprostheses calculate stimulation patterns to produce desired arm movement. However, currently available FES controllers have yet to restore natural movements. Reinforcement learning (RL) is a reward-driven control technique; it can employ user-generated rewards, and human preferences can be used in training. To test this concept with FES, we conducted simulation experiments using computer-generated ``pseudohuman{\u27\u27} rewards. Rewards with varying properties were used with an actor-critic RL controller for a planar two-degree-of-freedom biomechanical human arm model performing reaching movements. Results demonstrate that sparse, delayed pseudo-human rewards permit stable and effective RL controller learning. The frequency of reward is proportional to learning success, and human-scale sparse rewards permit greater learning than exclusively automated rewards. Diversity of training task sets did not affect learning. Longterm stability of trained controllers was observed. Using human-generated rewards to train RL controllers for upper-extremity FES systems may be useful. Our findings represent progress toward achieving human-machine teaming in control of upper-extremity FES systems for more natural arm movements based on human user preferences and RL algorithm learning capabilities

Training an Actor-Critic Reinforcement Learning Controller for Arm Movement Using Human-Generated Rewards

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Functional Electrical Stimulation (FES) employs neuroprostheses to apply electrical current to the nerves and muscles of individuals paralyzed by spinal cord injury (SCI) to restore voluntary movement. Neuroprosthesis controllers calculate stimulation patterns to produce desired actions. To date, no existing controller is able to efficiently adapt its control strategy to the wide range of possible physiological arm characteristics, reaching movements, and user preferences that vary over time. Reinforcement learning (RL) is a control strategy that can incorporate human reward signals as inputs to allow human users to shape controller behavior. In this study, ten neurologically intact human participants assigned subjective numerical rewards to train RL controllers, evaluating animations of goal-oriented reaching tasks performed using a planar musculoskeletal human arm simulation. The RL controller learning achieved using human trainers was compared with learning accomplished using human-like rewards generated by an algorithm; metrics included success at reaching the specified target; time required to reach the target; and target overshoot. Both sets of controllers learned efficiently and with minimal differences, significantly outperforming standard controllers. Reward positivity and consistency were found to be unrelated to learning success. These results suggest that human rewards can be used effectively to train RL-based FES controllers.NIH #TRN030167Veterans Administration Rehabilitation Research & Development predoctoral fellowshipArdiem Medical Arm Control Device grant #W81XWH072004

Sampling-based Algorithms for Optimal Motion Planning

During the last decade, sampling-based path planning algorithms, such as Probabilistic RoadMaps (PRM) and Rapidly-exploring Random Trees (RRT), have been shown to work well in practice and possess theoretical guarantees such as probabilistic completeness. However, little effort has been devoted to the formal analysis of the quality of the solution returned by such algorithms, e.g., as a function of the number of samples. The purpose of this paper is to fill this gap, by rigorously analyzing the asymptotic behavior of the cost of the solution returned by stochastic sampling-based algorithms as the number of samples increases. A number of negative results are provided, characterizing existing algorithms, e.g., showing that, under mild technical conditions, the cost of the solution returned by broadly used sampling-based algorithms converges almost surely to a non-optimal value. The main contribution of the paper is the introduction of new algorithms, namely, PRM* and RRT*, which are provably asymptotically optimal, i.e., such that the cost of the returned solution converges almost surely to the optimum. Moreover, it is shown that the computational complexity of the new algorithms is within a constant factor of that of their probabilistically complete (but not asymptotically optimal) counterparts. The analysis in this paper hinges on novel connections between stochastic sampling-based path planning algorithms and the theory of random geometric graphs.Comment: 76 pages, 26 figures, to appear in International Journal of Robotics Researc

Multi-agent Poli-RRT* Optimal constrained RRT-based planning for multiple vehicles with feedback linearisable dynamics

Planning a trajectory that is optimal according to some performance criterion, collision-free, and feasible with respect to dynamic and actuation constraints is a key functionality of an autonomous vehicle. Poli-RRT* is a sample-based planning algorithm that serves this purpose for a single vehicle with feedback linearisable dynamics. This paper extends Poli-RRT* to a multi-agent cooperative setting where multiple vehicles share the same environment and need to avoid each other besides some static obstacles

A Survey on Continuous Time Computations

We provide an overview of theories of continuous time computation. These theories allow us to understand both the hardness of questions related to continuous time dynamical systems and the computational power of continuous time analog models. We survey the existing models, summarizing results, and point to relevant references in the literature